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Abstract. Recent experimental work has suggested that the neural firing rate can be interpreted 
as a fractional derivative, at least when signal variation induces neural adaptation. Here, we show 
that the actual neural spike-train itself can be considered as the fractional derivative, provided 
that the neural signal is approximated by a sum of power-law kernels. A simple standard thresh- 
olding spiking neuron suffices to carry out such an approximation, given a suitable refractory 
response. Empirically, we find that the online approximation of signals with a sum of power-law 
kernels is beneficial for encoding signals with slowly varying components, like long-memory 
self-similar signals. For such signals, the online power-law kernel approximation typically re- 
quired less than half the number of spikes for similar SNR as compared to sums of similar but 
exponentially decaying kernels. As power-law kernels can be accurately approximated using 
sums or cascades of weighted exponentials, we demonstrate that the corresponding decoding of 
spike-trains by a receiving neuron allows for natural and transparent temporal signal filtering by 
tuning the weights of the decoding kernel. 

1 Introduction 

A key issue in computational neuroscience is the interpretation of neural signaling, as expressed by 
a neuron's sequence of action potentials. An emerging notion is that neurons may in fact encode 
information at multiple timescales simultaneously 111213141 : the precise timing of spikes may be con- 
veying high-frequency information, and slower measures, like the rate of spiking, may be relating 
low-frequency information. Such multi-timescale encoding comes naturally, at least for sensory neu- 
rons, as the statistics of the outside world often exhibit self-similar multi-timescale features J5] and 
the magnitude of natural signals can extend over several orders. Since neurons are limited in the rate 
and resolution with which they can emit spikes, the mapping of large dynamic-range signals into 
spike-trains is an integral part of attempts at understanding neural coding. 

Experiments have extensively demonstrated that neurons adapt their response when facing persis- 
tent changes in signal magnitude. Typically, adaptation changes the relation between the magnitude 
of the signal and the neuron's discharge rate. Since adaptation thus naturally relates to neural coding, 
it has been extensively scrutinized [67 8]. Importantly, adaptation is found to additionally exhibit fea- 
tures like dynamic gain control, when the standard deviation but not the mean of the signal changes 
[ 1 1, and long-range time-dependent changes in the spike -rate response are found in response to large 
magnitude signal steps, with the changes following a power-law decay (e.g. |9l). 

Tying the notions of self-similar multi-scale natural signals and adaptive neural coding together, 
it has recently been suggested that neuronal adaptation allows neuronal spiking to communicate a 
fractional derivative of the actual computed signal 110141 . Fractional derivatives are a generalization 
of standard 'integer' derivatives ('first order', 'second order'), to real valued derivatives (e.g. '0.5th 
order'). A key feature of such derivatives is that they are non-local, and rather convey information 
over essentially a large part of the signal spectrum 1 10 1. 

Here, we show how neural spikes can encode temporal signals when the spike-train itself 'is taken 
as the fractional derivative of the signal. We show that this is the case for a signal approximated 
by a sum of shifted power-law kernels starting at respective times ti and decaying proportional to 



— 1») . Then, the fractional derivative of this approximated signal corresponds to a sum of spikes 
at times provided that the order of fractional differentiation a is equal to 1 — j3: a spike-train is the 
a = 0.2 fractional derivative of a signal approximated by a sum of power-law kernels with exponent 
/3 = 0.8. Such signal encoding with power-law kernels can be carried out for example with simple 
standard thresholding spiking neurons with a refractory reset following a power-law. 

As fractional derivatives contain information over many time-ranges, they are naturally suited 
for predicting signals. This links to notions of predictive coding, where neurons communicate devi- 
ations from expected signals rather than the signal itself. Predictive coding has been suggested as a 
key feature of neuronal processing in e.g. the retina ifTTI . For self-similar scale-free signals, future 
signals may be influenced by past signals over very extended time-ranges: so-called long-memory. 
For example, fractional Brownian motion (ffim) can exhibit long-memory, depending on their Hurst- 
parameter H. For H > 0.5 fBM models which exhibit long-range dependence (long-memory) where 
the autocorrelation-function follows a power-law decay |[T2l . The long-memory nature of signals 
approximated with sums of power-law kernels naturally extends this signal approximation into the 
future along the autocorrelation of the signal, at least for self-similar l// 7 like signals. The key "pre- 
dictive" assumption we make is that a neuron's spike-train up to time t contains all the information 
that the past signal contributes to the future signal t' > t. 

The correspondence between a spike-train as a fractional derivative and a signal approximated as 
a sum of power-law kernels is only exact when spike-trains are taken as a sum of Dirac-<5 functions 
and the power-law kernels as l/i* 3 . As both responses are singular, neurons would only be able to 
approximate this. We show empirically how sums of (approximated) 1/t" power-law kernels can 
accurately approximate long-memory fBm signals via simple difference thresholding, in an online 
greedy fashion. Thus encodings signals, we show that the power-law kernels approximate synthe- 
sized signals with about half the number of spikes to obtain the same Signal-to-Noise-Ratio, when 
compared to the same encoding method using similar but exponentially decaying kernels. 

We further demonstrate the approximation of sine wave modulated white-noise signals with sums 
of power-law kernels. The resulting spike-trains, expressed as "instantaneous spike-rate", exhibit the 
phase-presession as in [4|, with suppression of activity on the "back" of the sine-wave modulation, 
and stronger suppression for lower values of the power-law exponent (corresponding to a higher 
order for our fractional derivative). We find the effect is stronger when encoding the actual sine wave 
envelope, mimicking the difference between thalamic and cortical neurons reported in [4|. This may 
suggest that these cortical neurons are more concerned with encoding the sine wave envelope. 

The power-law approximation also allows for the transparent and straightforward implementa- 
tion of temporal signal filtering by a post-synaptic, receiving neuron. Since neural decoding by a 
receiving neuron corresponds to adding a power-law kernel for each received spike, modifying this 
receiving power-law kernel then corresponds to a temporal filtering operation, effectively exploiting 
the wide-spectrum nature of power-law kernels. This is particularly relevant, since, as has been am- 
ply noted 091131 . power-law dynamics can be closely approximated by a weighted sum or cascade of 
exponential kernels. Temporal filtering would then correspond to simply tuning the weights for this 
sum or cascade. We illustrate this notion with an encoding/decoding example for both a high-pass 
and low-pass filter. 

2 Power-law Signal Encoding 

Neural processing can often be reduced to a Linear-Non-Linear (LNL) filtering operation on incom- 
ing signals [ 14 1 (figureQ}, where inputs are linearly weighted and then passed through a non-linearity 
to yield the neural activation. As this computation yields analog activations, and neurons communi- 
cate through spikes, the additional problem faced by spiking neurons is to decode the incoming signal 
and then encode the computed LNL filter again into a spike-train. The standard spiking neuron model 
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Fig. 1. Linear-Non-Linear filter, with spike-decoding front-end and spike-encoding back-end. 



is that of Linear-Nonlinear-Poisson spiking, where spikes have a stochastic relationship to the com- 
puted activation 1131 . Here, we interpret the spike encoding and decoding in the light of processing 
and communicating signals with fractional derivatives [ 10|. 

At least for signals with mainly (relatively) high-frequency components, it has been well estab- 
lished that a neural signal can be decoded with high fidelity by associating a fixed kernel with each 
spike, and summing these kernels fl6l : keeping track of doublets and triplet spikes allows for even 
greater fidelity. This approach however only worked for signals with a frequency response lacking 
low frequencies 1 16 1. Low-frequency changes lead to "adaptation", where the kernel is adapted to fit 
the signal again 1171 . For long-range predictive coding, the absence of low frequencies leaves little 
to predict, as the effective correlation time of the signals is then typically very short as well 1 16|. 

Using the notion of predictive coding in the context of (possible) long-range dependencies, we 
define the goal of signal encoding as follows: let a signal xj(t) be the result of the continuous-time 
computation in neuron j up to time t, and let neuron j have emitted spikes tj up to time t. These 
spikes should be emitted such that the signal Xj{t') for t' < t is decoded up to some signal-to-noise 
ratio, and these spikes should be predictive for Xj (t') for t' > t in the sense that no additional spikes 
are needed at times t' > t to convey the predictive information up to time t. 

Taking kernels as a signal filter of fixed width, as in the general approach in [ 16 1 has the important 
drawback that the signal reconstruction incurs a delay for the duration of the filter: its detection cannot 
be communicated until the filter is actually matched to the signal. This is inherent to any backward- 
looking filter-maching solution. Alternatively, a predictive coding approach could rely on only on a 
very short backward looking filter, minimizing the delay in the system, and continuously computing 
a forward predictive signal. At any time in the future then, only deviations of the actual signal from 
this expectation are communicated. 



2.1 Spike-trains as fractional derivative 

As recent work has highlighted the possibility that neurons encode fractional derivatives, it is note- 
worthy that the non-local nature of fractional calculus offers a natural framework for predictive cod- 
ing. In particular, as we will show, when we assume that the predictive information about the future 
signal is fully contained in the current set of spikes, a signal approximated as a sum of power-law 
kernels corresponds to a fractional derivative in the form of a sum of Dirac-<5 functions, which the 
neuron can obviously communicate through timed spikes. 

The fractional derivative r(t) of a signal x(t) is denoted as D a x(t), and intuitively expresses: 



where a is the fractional order, e.g. 0.5. This is most conveniently computed through the Fourier 
transformation in the frequency domain, as a simple multiplication: 

R{uj) = H{lo)X{uj), 

where the Fourier-transformed fractional derivative operator H(oj) is by definition (iui) a iflOl . and 
X{u}) and R(ui) are the Fourier transforms of x(t) and r(t) respectively. 

We assume that neurons carry out predictive coding by emitting spikes such that all predictive 
information is contained in the current spikes, and no more spikes will be fired if the signal follows 
this prediction. Approximating spikes by Dirac-<5 functions, we take the spike-train up to some time 
to to be the fractional derivative of the past signal and be fully predictive for the expected influence 
the past signal has on the future signal: 

r(i) = J2 *(*"*0 

U<t 

The task is to find a signal x(t) that corresponds to an approximation of the actual signal x(t) up to 
to, and where the predicted signal contribution x(t) for t > to due to x(t < to) does not require 
additional future spikes. We note that a sum of power-law decaying kernels with power-law t~@ for 
ft = 1 — a corresponds to such a fractional derivative: the Fourier- transform for a power-law decaying 
kernel of form t~P is proportional to (iuj) 13 ^ 1 , hence for a signal that just experienced a single step 
from to 1 at time t we get: 

R(u) = M Q M' 3 - 1 , 

and setting (3 = 1 — a yields a constant in Fourier-space, which of course is the Fourier-transform 
of S(t). It is easy to check that shifted power-law decaying kernels, e.g. (t — f a ) correspond to 
a shifted fractional derivative 5(t — t a ), and the fractional derivative of a sum of shifted power-law 
decaying kernels corresponds to a sum of shifted delta-functions. Note that for decaying power-laws, 
we need (3 > 0, and for fractional derivatives we require a > 0. 

Thus, with the reverse reasoning, a signal approximated as the sum of power-law decaying kernels 
corresponds to a spike-train with spikes positioned at the start of the kernel, and, beyond a current 
time t, this sum of decaying kernels is is interpreted as a prediction of the extent to which the future 
signal can be predicted by the past signal. 

Obviously, both the Dirac-<5 function and the 1 ft" kernels are singular (figure^) and can only be 
approximated. For real applications, only some part of the 1/t" curve can be considered, effectively 
leaving the magnitude of the kernel and the high frequency component (the extend to which the initial 
l/t° peak is approximated) as free parameters. Figure [2J3 illustrates the signal approximated by a 
random spikes train; as compared to a sum of exponentially decaying a-kernels, the long-memory 
effects of power-law decay kernels is evident. 

2.2 Practical encoding 

To explore the efficacy of the power-law kernel approach to signal encoding/decoding, we take a 
standard thresholding online approximation approach, where neurons communicate only deviations 
between the current computed signal x(t) and the emitted approximated signal x(t) exceeding some 
threshold 9. The emitted signal x(t) is constructed as the (delayed) sum of filter kernels k each 
starting at the time of the emitted spike: 

x(t) = £ K(t - {tj + A)), 

tj<t 

the delay A corresponds to the time-window over which the neuron considers the difference between 
computed and emitted signal. In a spiking neuron, such computation would be implemented simply 




Fig. 2. a) Signal x(t) and corresponding fractional derivative r(t): 1/t" power-laws and delta- 
functions; b) power-law approximation, timed to spikes; compared to sum of a-functions (black 
dashed line), c) Approximated l/i* 3 power-law kernel for different values of k from eq. (2). d) The 
approximated power-law kernel (blue line) can be decomposed as a weighted sum of a-functions 
with various decay time-constants (dashed lines). 



by for instance a refractory current following a power-law. Allowing for both positive and negative 
spikes (corresponding to tightly coupled neurons with reversed threshold polarity |16|), this would 
expand to: 

t+<t t-<t 

Considering just the fixed time-window thresholding approach, a spike is emitted each time the dif- 
ference between the computed signal x(t) and the emitted signal x(t) plus (or minus) the kernel K(t) 
summed over some time-window exceeds the threshold 8: 

to 

r(t )=5(t ) if \x(t)-x(t)\-\x(t)-(x(t)+k(t))\)>6, 

r=t -A 
to 

= -S(t ) if J2 \ X ( T ) ~ ~ \x(t) - (x(t) - k(t))\) >6, (1) 

T=t -A 

the signal approximation improvement is computed here as the absolute value of the difference be- 
tween the current signal noise and the signal noise when a kernel is added (or subtracted). 

As an approximation of 1 jt$ power-law kernels, we let the kernel first quickly rise, and then 
decay according to the power-law. For a practical implementation, we use a 1/t 13 signal multiplied 
by a modified version of the logistic sigmoid function logsig(<) = 1/(1 + exp(— t)): v(t,k) — 
2 logsig(fct) — 1, such that the kernel becomes: 



K(t) = Xv(t,k)l/t p , 



(2) 



where n(t) is zero for t' < t, and parameter k determines the angle of the initial increasing part of the 
kernel. The resulting kernel is further scaled by a factor A to achieve a certain signal approximation 
precision (kernels for power-law exponential f3 = 0.5 and several values of k are shown in figure[2j;). 
As an aside, the resulting (normalized) power-law kernel can very accurately be approximated over 
multiple orders of magnitude by a sum of just 1 1 a-function exponentials (figure[2jl). 

Next, we compare the efficiency of signal approximation with power-law predictive kernels as 
compared to the same approximation using standard fixed kernels. For this, we synthesize self-similar 
signals with long-range dependencies. We first remark on some properties of self-similar signals with 
power-law statistics, and on how to synthesize them. 

2.3 Self-similar signals with power-law statistics 

There is extensive literature on the synthesis of statistically self-similar signals with 1 / /-like statis- 
tics, at least going back to Kolmogorov [ 18 1 and Mandelbrot [ 19 1. Self-similar signals exhibit slowly 
decaying variances, long-range dependencies and a spectral density following a power law. Im- 
portantly, for wide-sense self-similar signals, the autocorrelation functions also decays following a 
power-law. Although various distinct classes of self-similar signals with 1 / /-like statistics exist fl2l . 
fractional Brownian motion (ffim) is a popular model for many natural signals. Fractional Brownian 
motion is characterized by its Hurst-paramater H, where H = 0.5 corresponds to regular Brownian 
motion, and fBM models with H > 0.5 exhibit long-range (positive) dependence. The spectral den- 
sity of an fBm signal is proportional to a power-law, l// 7 , where 7 = 2H + 1. We used fractional 
Brownian motion to generate self-similar signals for various H values, using the wf bm function from 
the Matlab wavelet toolbox. 

3 Signal encoding/decoding 

3.1 Encoding long-memory self-similar signals 

We applied the thresholded kernel approximation outlined above to synthesized fBm signals with 
H > 0.5, to ensure long-term dependence in the signal. An example of such encoding is given in 
figure [3] left panel, using both positive and negative spikes, (inset, red line: the power-law kernel 
used). When encoding the same signal with kernels without the power-law tail (inset, blue line), the 
approximation required more than twice as many spikes for the same Signal-to-Noise-Ratio (SNR). 

In figure[3] right panel, we compared the encoding efficacy for signals with different H -parameters, 
as a function of the power-law exponent, using the same number of spikes for each signal (achieved 
by changing the A parameter and the threshold 9). We find that more slowly varying signals, corre- 
sponding to higher i7-parameters, are better encoded by the power-law kernels, More surprisingly, 
we find and signals are consistently best encoded for low /3-values, in the order of 0.1 — 0.3. Similar 
results were obtained for different values of k in equation ©. 

We should remark that without negative spikes, there is no longer a clear performance advantage 
for power-law kernels (even for large /?): where power-law kernels are beneficial on the rising part of 
a signal, they lose on downslopes where their slow decay cannot follow the signal. 

3.2 Sine-wave modulated white-noise 

Fractional derivatives as an interpretation of neuronal firing-rate has been put forward by a series of 
recent papers 1 10 20 4|, where experimental evidence was presented to suggest such an interpreta- 
tion. A key finding in |4) was that the instantaneous firing rate of neurons along various processing 
stages of a rat's whisker movement exhibit a phase-lead relative to the amplitude of the movement 
modulation. The phase-lead was found to be greater for cortical neurons as compared to thalamic 
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Fig. 3. Left: example of encoding of ffim signal with power-law kernels. Using an exponentially 
decaying kernel (inset) required 1398 spikes vs. 618 for the power-law kernel (k = 50), for the same 
SNR. Right: SNR for various (3 power-law exponents using a fixed number of spikes (48Hz), with 
curves for different H -parameters, each curve averaged over five 16s signals. The dashed blue curve 
plots the H = 0.6 curve, using less spikes (36Hz); the fiat bottom dotted line shows the average 
performance of the non-power-law exponentially decaying kernel, also for H = 0.6. 

neurons. When the firing rate corresponds to the a-order fractional derivative, the phase-lead would 
correspond to greater fractional order a in the cortical neurons iflOl . We used the sum-of-power-laws 
to approximate both the sine-wave-modulated white noise and the actual sine-wave itself, and found 
similar results (figure 3J: smaller power-law exponents, in our interpretation also corresponding to 
larger fractional derivative orders, lead to increasingly fewer spikes at the back of the sine-wave 
(both in the case where we encode the signal with both positive and negative spikes - then counting 
only the positive spikes - and when the signal is approximated with only positive spikes - not shown). 
We find an increased phase-lead when approximating the actual sine-wave kernel as opposed to the 
white-noise modulation, suggesting that perhaps cortical neurons more closely encode the former as 
compared to thalamic neurons. 

3.3 Signal Frequency Filtering 

For a receiving neuron i to properly interpret a spike-train r(t)j from neuron j, both neurons would 
need to keep track of past events over extended periods of time: current spikes have to be added to or 
subtracted from the future expectation signal that was already communicated through past spikes. The 
required power-law processes can be implemented in various manners, for instance as a weighted sum 
or a cascade of exponential processes H9I101 . A natural benefit of implementing power-law kernels as 
a weighted sum or cascade of exponentials is that a receiving neuron can carry out temporal signal 
filtering simply by tuning the respective weight parameters for the kernel with which it decodes spikes 
into a signal approximation. 

In figure[5] we illustrate this with power-law kernels that are transformed into high-pass and low- 
pass filters. We first approximated our power-law kernel d2) with a sum of 1 1 exponentials (depicted 
in the left-center inset). Using this approximation, we encoded the signal (figure[5] center). The signal 
was then reconstructed using the resultant spikes, using the power-law kernel approximation, but with 
some zeroed out exponentials (respectively the slowly decaying exponentials for the high-pass filter, 
and the fast-decaying kernels for the low-pass filter). Figure|5] most right, shows the resulting filtered 
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Fig. 4. Sinewave phase-lead. Left: when encoding sine-wave modulated white noise (inset); right: en- 
coding the sine-wave signal itself (inset). Average firing rate is computed over 100ms, and normalized 
to match the sine-wave kernel. 
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Fig. 5. Illustration of frequency filtering with modified decoding kernels. The square boxes show the 
respective kernels in both time and frequency space. See text for further explanation. 

signal approximations. Obviously, more elaborate tuning of the decoding kernel with a larger sum of 
kernels can approximate a vast variety of signal filters. 

4 Discussion 

Taking advantage of the relationship between power-laws and fractional derivatives, we outlined the 
peculiar fact that a sum of Dirac-r5 functions, when taken as a fractional derivative, corresponds to 
a signal in the form of a sum of power-law kernels. Exploiting the obvious link to spiking neural 
coding, we showed how a simple thresholding spiking neuron can compute a signal approximation 
as a sum of power-law kernels; importantly, such a simple thresholding spiking neuron closely fits 
standard biological spiking neuron models, when the refractory response follows a power-law de- 
cay (e.g. ET|). We demonstrated the usefulness of such an approximation when encoding slowly 
varying signals, finding that encoding with power-law kernels significantly outperformed similar but 
exponentially decaying kernels that do not take long-range signal dependencies into account. 



Compared to the work where the firing rate is considered as a fractional derivative, e.g. ifTUll . 
the present formulation extends the notion of neural coding with fractional derivatives to individual 
spikes, and hence finer temporal variations: each spike effectively encodes very local signal varia- 
tions, while also keeping track of long-range variations. 

The interpretation in [ 10] of the fractional derivative r(t) as a rate leads to a 1:1 relation between 
the fractional derivative order and the power-law decay exponent of adaptation of about 0.2 M10I22I9L 
For such fractional derivative a, our derivation implies a power-law exponent for the power law 
kernels j3 = 1 — a w 0.8, consistent with our sine-wave reconstruction, as well as with recent adapting 
spiking neuron models ETTl . We find that when signals are approximated with non-coupled positive 
and negative neurons (i.e. one neuron encodes the positive part of the signal, the other the negative), 
such much faster-decaying power-law kernels encode more efficiently than slower decaying ones. 
Non-coupled signal encoding obviously fair badly when signals rapidly change polarity; this however 
seems consistent with human illusory experiences l23ll . 

As noted, the singularity of 1/t 13 power-law kernels means that initial part of the kernel can 
only be approximated. Here, we initially focused our simulation on the use of long-range power-law 
kernels for encoding slowly varying signals. A more detailed approximation of this initial part of 
the kernel may be needed to incorporate effects like gain modulation [24 8 1, and determine up to 
what extent the power-law kernels already account for this phenomenon. This would also provide a 
natural link to existing neural models of spike-frequency adaptation, e.g. |25l . as they are primarily 
concerned with modeling the spiking neuron behavior rather than the computational aspects. 

We used a greedy online thresholding process to determine when a neuron would spike to approx- 
imate a signal, this in contrast to offline optimization methods that place spikes at optimal times, like 
Smith & Lewicki |26|. The key difference of course is that the latter work is concerned with decod- 
ing a signal, and in effect attempts to determine the effective neural (temporal) filter. As we aimed to 
illustrate in the signal filtering example, these notions are not mutually exclusive: a receiving neuron 
could very well filter the incoming signal with a carefully shaped weighted sum of kernels, and then, 
when the filter is activated, signal the magnitude of the match through fractional spiking. 

Predictive coding seeks to find a careful balance between encoding known information as well 
as future, derived expectations |27|. It does not seem unreasonable to formulate this balance as a no- 
going-back problem, where current computations are projected forward in time, and corrected where 
needed. In terms of spikes, this would correspond to our assumption that, absent new information, no 
additional spikes need to be fired by a neuron to transmit this forward information. 

The kernels we find are somewhat in contrast to the kernels found by Bialek et. al. fl6l . where the 
optimal filter exhibited both a negative and a positive part and no long-range "tail". Several practical 
issues may contribute to this difference, not least the relative absence of low frequency variations, as 
well as the fact that the signal considered is derived from the fly's HI neurons. These two neurons 
have only partially overlapping receptive fields, and the separation into positive and negative spikes is 
thus slightly more intricate. We need to remark though that we see no impediment for the presented 
signal approximation to be adapted to such situations, or situations where more than two neurons 
encode fractions of a signal, as in population coding, e.g. 11281 . 

Finally, we would like to remark that the issue of long-range temporal dependencies such as 
discussed here seems to be relatively unappreciated. As pointed out in 0, long-range power-law 
dynamics would seem to offer a variety of "hooks" for computation through time, like for tempo- 
ral difference learning and relative temporal computations (and possibly exploiting the many noted 
correspondences between spatial and temporal statistics |29l ). 
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